A Nested Hidden Markov Model for Internet Browsing Behavior

Internet browsers generate “click stream” data that supply information about the path taken through a web site. Click stream data have an interesting temporal structure in which sequences of individual page requests are nested within browsing sessions, and some users return to the web site for multiple sessions. We model sequences of page requests within a session using a hidden Markov mixture of first order Markov chains. A second hidden Markov chain describes variation between consecutive web sessions by the same user. Forward-backward recursions can be employed to simultaneously draw both latent Markov chains directly from their joint posterior distribution. The recursions make MCMC posterior simulation particularly efficient. The model allows probabilities of interest, such as the probability that a web session contains a purchase, to be computed directly from model parameters without resorting to Monte Carlo integration. We apply the model to data collected from a small e-commerce site and find that the session-level and page-level hidden Markov chains play different roles. The session-level model discovers a set of “session types” that correspond to recognizable patterns of user behavior. The page-level models account for data features that would be poorly described by first order Markov chains. The flexible page-level models allow the session-level model to describe the data with relatively few session types.